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© Method and apparatus for forming a video signal using motion estimation and signal paths with 
different Inteipolation processing. 



© In motion cornpensated field rate upconversion 
systems intermediate fieWs of a television picture, 
not prmBent" Mj;i|ie\\inS^ut- A. are calculated by rneans 
of motion compensated interpolation. A typical ap- 
plication is conversion between 50 Hz and 100 Hz 
television signals for display flicker reduction. In 
principle, every pixel in the intermediate fields can 
be the direct output of a motion compensated inter- 
polator (MCI). In practice, however, there will always 
be areas of the picture where the motion is too fast 
or too compjex to be estimated correctly. 
According to the itwev^ ^e ^r^ output (O) is in 
fact the result of a soft switch (SSW). or fade be- 
tween the motion compensated interpolation (MCI) 
and a fallback mode (FR), controlled by a measure 
of confidence (CF) in the reliability of the estimated 
motion vector. Advantageously, the confidence mea- 
sure includes the following two components: a "basic 
confidence measure* which reflects the error (BME) 
in the motion measwenwnt. and a measure based 
on the speed of moyeirieht (MVX, MVY). 
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The present invention relates to a method and 
to an apparatus for forming a video signal using 
motion estimation and signal paths with different 
interpolation processing. 

Background 

The invention described here is part of a mo- 
tion compensated field rate upconversion system, 
where intermediate fields of a television picture, not 
present at the input are calculated by means of 
motion compensated interpolation. A typical ap- 
plication is conversion between 50 Hz and 100Hz 
television signals for display flicker reduction. 

Invention 

It is one object of the invention to disclose a 
method of selecting an interpolation mode in rela- 
tion to ; a confidence measurement of the related 
motion vector. This object is reached by the meth- 
od disclosed in claim 1 . 

It is a further object of the invention to disclose 
an apparatus which utilises the inventive method. 
This object is reached by the apparatus disclosed 
' in clairh 

in ■. principle, every pixel in the intermediate 
fields can be the direct output of a motion com- 
pensated interpolator. In practice, however, there 
will always be areas of the picture where the mo- 
tion is too fast or too complex to be estimated 
correcUy. Patent application EP93 402505 of the 
applicant describes *f aiback processing ' whereby a 
special non-compensated fallback mode or any 
comparable mode can be used in such areas. 

The final output is in fact the result of a soft 
switch or fade between the motion compensated 
interpolation and the fallback mode, controlled by a 
measure of confidence in the reliability of the es- 
ti mated motion vector. The confidence measure 
takes, values between 0 and 1, where 0 (no con- 
fidence) selects the fallback mode only, 1 (ma»- 
m urn confidence) selects the motion compensated 
interpolation brily, and intermediate values mix lin- 
early between the two. AoVantageously, the con- 
fidence measure includes the following two compo- 
nents: a *bastc confidence measure* which reflects 
the error irr ^molipn^ mfl^.remenl f . and ' a mea- 
sure ba^ on ^ speed of movement 

In principled the inventive method is suited for 
forming a video signal using motion estimation and 
signal paths with different interpolation processing, 
in particular motion compensated interpolation and 
fallback interpolation, whereby the output signals of 
said different signal paths are combined in relation 
to a measure of confidence which is derived from a 
minimum motion estimation error of the input video 
signal, which is in particular block based. 



Advantageous additional embodiments of the 
inventive method are resulting from the respective 
dependent clams. 

In principle the inventive apparatus for forming 
5 a video signal using motion estimation and signal 
paths with different interpolation processing, in par- 
ticular motion compensated interpolation and fall- 
back interpolation, includes: 

- motion compensated interpolation means and 
io fallback interpolation means which receive 

said input video signal and the output pixel 
values of which are mixed in combining 
means in relation to a measure of confidence 
value which is derived in combining means 
rs from a block based minimum motion estima- 

tion enor of the input video signal and from a 
related block motion vector or its compo- 
nents; 

- motion estimation means which calculate 
20 from said input signal, or motion information 

extractor means which extract from the trans- 
mitted input signal data stream, said block 
based minimum motion estimation error and 
said related block motion vector or its com- 
as ponehts. 

Advantageous additional embodiments of the 
inventive apparatus are resulting from the respec- 
tive dependent claims. 

30 Drawings 

Preferred embodiments of the invention are 
described with reference to the accompanying 
drawings, which show in: 
as fig. 1 example of law reiati ng confidence to 
sunblock matching error; 
Fig. 2 example of law relating confidence to 

estimated speed; 
Fig. 3 block diagram of confidence measure- 
40 rnent circuit: 

Preferred embodiments 

One method of motion measurement is block 
45 matching followed by localization using subbtock 
matching, as described in EP-A-93402188 of the 
applicant. 

Such subbtock matching process produces motion 
vectors whose components may be passed through 

so a median fitter. In calculating the motion vectors, 
the subblock matching process or any other com- 
parable process generates an error {e.g. the sum of 
absolute, or the square of, difference values be- 
tween the according pixel values in the two blocks 

55 used for matching) for each candidate motion vec- 
tor and the vector giving the minimum error is 
chosen. 

It is the value of this minimum error which is used 
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to calculate a basic measure of confidence accord- 
ing to the law illustrated in fig. 1. If this subblock 
matching error SBME is zero, the confidence CF 
takes its maximum value of 1. As the error SBME 
increases to a cut-off value CV t the confidence 5 
decreases to its minimum va*ue of zero. Other, 
similar relations (e.g. part of a sine or cosine func- 
tion or a gaussian function) between subblock 
matching error and confidence may also be used 
for such limiting filter. 10 
One problem with this basic measure of confidence 
is that the subblock matching process usually gen- 
erates high errors, leading to low or zero con- 
fidence, around the edge of an object moving over 
a detailed background. This means that the edge is 
will be interpolated using the fallback mode., even 
though the motion vectors themselves correctly 
follow the edge. A solution is to pass the subblock 
errors through a rank order fitter, for example on a 
3x3 window. The filter may be a median filter or 20 
another rank order filter; for example the third high- 
est error value in the window may be selected. 

Advantageously, the rneasure of confidence 
may be modified to take account of the speed ES 
of movement. The aim , of this modification is to 25 
ensure that, as an object accelerates out of the 
range of motion speeds that can be estimated and 
coded with a motion vector, the interpolation 
switches smoothly into the faffi>ack mode. An ex- 
ample of a suitable law that can be applied is given 30 
in fig. 2. Again, other relations (e.g. part of a sine 
or cosine function or a gaussian function, different 
start value SV) between motion speed ES and 
confidence CF may be used. 

It can be applied to the true motion speed or to the 35 
two motion vector cpmpbrients sepajatery, taking 
the minimum or some other combination of the two 
results the result is combined with the main mea- 
sure of confidence, for example by taking the mini- 
mum or by multiplication. 40 

In the confidence measurement circuit of Fig. 3 
an input video signal, e.g. 50Hz. 2:1 or 60Hz. 2:1, 
is fed to a motion compensated interpolator MCI, to 
a fallback filter FFI and to a motion estimator ME. 
A soft switch SSW operates on the output signals 45 
of interpolator MCI and filter FFI and provides the 
final output signal O which may have the format 
1007120Hz, 2:1 or 50>60Hz. 1:1. Switch SSW is 
controlled by a confidence signal CF coming from 
a combiner CB. Motion estimator ME calculates the so 
block matching error BME, a horizontal component 
MVX and a vertical component MVY of the related 
motion vector. These values can also be transmit- 
ted if the motion estimator is located within a 
transmitter and the circuit of fig. 3 is part of a 55 
receiver. e.g. a TV receiver. Then, circuit ME may 
extract the matching error and/or the motion vector 
and/or its components from the transmitted input 



signal data stream. The motion information can 
also be used in interpolator MCI. Error BME passes 
through a rank order filter ROFI. for instance a 
median filter, and a first weighting filter L1 operat- 
ing according to the law of Rg. 1 and is then fed 
into a combiner CB. 

The horizontal vector component MVX passes 
through a first metfan filter MFI1 and a second 
weighting filter L2 operating according to the law of 
fig. 2 and is then fed into combiner CB. 
The vertical vector component MVY passes 
through a second median filter MFI2 and a third 
weighting filter L3 operating according to the law of 
fig. 2 and is then fed into combiner CB. 
The combiner CB operates as described above 
and can work as a look-up table which is stored in 
memory means. The laws of filters L2 and L3 can 
be different (different function or start value SV). 
The laws represented by filters LI, 12 and L3 can 
be stored in look-up tables. 

In case of specific picture data characteristics (e.g. 
camera/film mode) different laws can be selected 
or downloaded into the law memories. 
If the chrominance component^ are not controlled 
together with the luminance comjponents by signal 
CF, they can be handled by separate means ac- 
cording to Fig. 3, having laws designed especially 
for the chrominance components. 
The invention can also be used for standards con- 
version, bit rate reduction, digital HDTV, digital 
VCR and MPEG1 or MPEG2 devices (IEC/ISO 
standard). In case of bit rate reduction the fallback 
processing may be intra field coding. 

Claims 

1. Method for forming a video signal (O) using 
motion estimation (ME) and signal paths with 
different interpolation processing, in particular 
motion compensated interpolation (MCI) and 
fallback interpolation (FFI), characterised in 
that the output signals of said different signal 
paths are combined (SSW) in relation to a 
measure of confidence (CF) which is derived 
from a minimum motion estimation (ME) error 
(BME) of the input video signal (i), which is in 
particular block based. 

2. Method according to claim 1, characterised 
in that said measure of confidence (CF) is also 
derived from the value of a block motion vector 
(MVX, MVY). 

3. Method accordng to claim 2, characterised 
in that said measure of confidence (CF) is 
derived from the values of the horizontal (MVX) 
and vertical (MVY) components of said block 
motion vector. 
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4. Method according to any of claims 1 to 3. 
characterised In that the values of said mo- 
tion estimation error (BME) pass through two- 
dimensional rank order filter means (ROFI) be- 
fore being used to form (CB) said measure of 5 
confidence (CF). 

5. Method according to any of claims 2 to 4. 
characterised In that the values of said block 
motion vector or its components (MVX MVY) io 
pais through median filter means (MFI1 , MFI2) 
before being used to form said measure of 
confidence (CF), 

6. Method according to any of claims 1 to 5, is 
characterised In that the values or pre-pro- 
cessed valutas of said motion estimation error 
(BME) ahd/br said block motion vector or its 
components (MVX, MVY) pass through limiting 
filter means (L1.L2.L3) before being used to 20 
form said measure of confidence (CF). 

7. Metiiod ac<x>rding to any of claims 1 to 6, 
characterised in that from the values or pre- 
processed values of said motion estimation 25 
error (BME) and/or said block motion vector or 

its components (MVX, MVY) the minimum is 
taken, or these values are multiplied, to form 
said measure of confidence (CF). 
**«■■:■. "'y- 30 

a Apparatus for forming a video signal (O) using 
motion estimation (ME) and signal paths with 
different interpolation processing, in particular 
motion compensated interpolation (MCI) and 
fallback interpolation (FFI). related to a method 35 
according to any of claims 1 to 8 and indud- 

- motion compensated interpolation means 
(MCI) and fallback interpolation means 
(FFI) which receive said input video sig- *o 
naJ XO and the output pixel values of 
which are mixed in combining means 
(SSW) in relation to a measure of con- 
fioence value (CF> which is derived in 
combining means (CB) from a block 45 
based minimum motion estimation error 
(BME) of the input video signal (I) and 
from a related block motion vector or its 
components (MVX, MVY); 

- motion estimation means (ME) which cal- so 
culate from said input signal (I), or mo- 
tion information extractor means (ME) 
which extract from the transmitted input 
signal data stream, said block based 
minimum motion estimation error (BME) 55 
and said related block motion vector or 

its components (MVX. MVY). 



Apparatus according to claim 8. characteris- 
ed in that said block based minimum motion 
estimation error (BME) and said related block 
motion vector or its components (MVX MVY) 
pass through rank order or median filter means 
(ROFI. MFI1. MFI2) and/or through limiting fil- 
ter means (LI, L2. L3) before being used in 
said combining means (CB) to form said mea- 
sure of confidence value (CF). 
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© Method and apparatus for forming a video signal using motion estimation and signal paths with 
different Interpolation processing. 



© In motion compensated field rate upconversion 
systems intermediate fields of a television picture, 
not present at the input (I), are calculated by means 
of motion compensated interpolation. A typica! ap- 
plication is conversion between 50 Hz and 100 Hz 
television signals for display flicker reduction. In 
principle, every pixel in the intermediate fields can 
be the direct output of a motion compensated inter- 
polator (MCI). In practice, however, there will always 
be areas of the picture where the motion is too fast 
or too complex to be estimated correctly. 
According to the invention, the final output (O) is in 
fact the result of a soft switch (SSW) or fade be- 
tween the motion compensated interpolation (MCI) 
and a fallback mode (FFI), controlled by a measure 
of confidence (CF) in the reliability of the estimated 
motion vector. Advantageously, the confidence mea- 
sure includes the following two components: a 'basic 
confidence measure* which reflects the error (BME) 
in the motion measurement and a measure based 
on the speed of movement (MVX, MVY). 
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